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Abstract 

O 

q Commons-based peer production does seem to rest upon a paradox. Although 

CZ) users produce all contents, at the same time participation is commonly on a volun- 

rA tary basis, and largely incentivized by achievement of project's goals. This means 

O that users have to coordinate their actions and goals, in order to keep themselves 

' nJ from leaving. While this situation is easily explainable for small groups of highly 

^*~j committed, like-minded individuals, little is known about large-scale, heteroge- 

<**j neous projects, such as Wikipedia. 

Oh In this contribution we present a model of peer production in a large online 

community. The model features a dynamic population of bounded confidence 

I users, and an endogenous process of user departure. Using global sensitivity anal- 

^. ysis, we identify the most important parameters affecting the lifespan of user par- 

(*"•*) ticipation. We find that the model presents two distinct regimes, and that the shift 

£SJ between them is governed by the bounded confidence parameter. For low values 

\& of this parameter, users depart almost immediately. For high values, however, the 

l/~) model produces a bimodal distribution of user lifespan. These results suggest that 

i» > user participation to online communities could be explained in terms of group con- 

f *•) sensus, and provide a novel connection between models of opinion dynamics and 

—^ commons-based peer production. 

.O* 1 Introduction 

X 

In the past decade mass collaboration platforms have become common in several pro- 
duction contexts. The term commons-based peer production has been coined to refer 
to a broad range of collaborative systems, such as those used for producing software, 
sharing digital content, and organizing large knowledge repositories, however, seem 
to be based upon a paradox. In wikis, there is a link between quality and cooperation 
1 30 1, but, at the same time, contribution is voluntary, based on non-monetary incentives 



'Accepted to Socinfo 201 1. The final publication is available at www.springerlink.com 



For small teams, this might not be a problem. In large scale wikis, where low 
access barriers are necessary to attract vast masses of contributors [ 8 1, and where expert 
users play a crucial role in maintenance and governance [2 J, user retention becomes in- 
stead crucial [10|. 

An established fact about participation to online groups is the preferential behavior 
of users, that is, a newcomer's long-term participation can be predicted by the outcome 
of his or her early interactions iTTl ETIl . This could be explained in terms of Socialization 
theory |6], as users assess the willingness of the community to accept them and vice 
versa. It is also true, however, that quality assessment of the produced contents, and in 
particular comparison of the objectives of an individual with those of the community, 
is important in determining user participation [ 17 1. This could be explained as a form 
of day-to-day coordination or group consensus taking place among editors lfl5l . 

In this paper we study user participation as a collective social phenomenon |4|. 
Other models of peer-production have been proposed already, for example for social 
information filtering platforms |fT3l . Here, we draw specifically from the modeling 
work on models of social influence under bounded confidence ll9l [T2l . 

Let us consider a community of users engaged in editing a collection of pages, 
e.g. Wikipedia. Pages are denoted by a certain number of features upon which users 
can find themselves in agreement or not. For example, let us consider the writing 
style of pages. Users try to modify pages according to their objectives, i.e. using 
their own style. At the same time, by interacting with contents, users can be also 
influenced by the style of other users. This reciprocal influence, however, happens 
only to a certain extent, that is, only when user and page (that is, their styles) are 
similar enough. Vandals, to illustrate with the same example, might not be interested 
in learning the encyclopedic writing style. In the context of social psychology this 
phenomenon is known as bounded confidence, and is regarded as a general feature of 
human communication within groups that try to reach consensus fl2l . It can be also 
seen as a form of herding in that people are influenced by the social context they are in 
US). 

The population of users in our model is dynamic, with user departure determined 
endogenously by the social influence process. Although others have already studied 
Deffuant's model to a dynamic population |3|, here we explicitly link the process of 
social influence to user participation. 

We implemented these ideas in an agent-based model of a peer production system. 
In this model, several factors affect the behavior of agents, such as user activity, content 
popularity, and community growth. To understand what factors are truly important for 
the resulting dynamics of user participation, we performed a factor screening using 
global sensitivity analysis. 

1.1 Related work 

The subject of user participation in mass collaboration systems has been already touched 
by several authors, for example on social networking sites 1 16 1, and knowledge sharing 
platforms [3T). A "momentum" law has been proposed for the distribution of user life 
edits of inactive users [29]. The distribution of user account lifespans has been shown 
to decay with a heavy tail, and a power-law model has been proposed after this obser- 



vation 1 1 1 1. Empirical data from Wikipedia, however, seem to support a super-position 
of different regimes [ 7 1 ; a feature of the model we present here is indeed a bimodal dis- 
tribution of user lifespans. In the context of wikis and other free open source initiatives 
some authors have used survival analysis to outline the diffences between different 
communities, [20] but this modeling technique is not suited to understand the connec- 
tion between social influence, group coordination, and user retention. We advocate the 
need to explicitly model such processes explicitly. 

The paper is organized as follows: in Sec. |2]we introduce our model of peer produc- 
tion; in Sec. [5]we briefly describe global sensitivity analysis and Gaussian Processes, 
the two statistical techniques we used for the factor screening study; in Sec. fflwe 
present our main results and we discuss them in Sec. B] 

2 An agent-based model of commons-based peer pro- 
duction 

In this section we introduce our model of peer production. While we make explicit 
use of the terminology of wiki platforms (e.g. "users" who "edit pages") we stress that 
ours is a general model of consensus building in a dynamic bipartite population, and 
not merely a description of a wiki platform. We also stress that in our model the state 
of agents may not necessarily represent an opinion in the classic sense of other studies 
of opinion dynamics, i.e. extremes of the spectrum do not necessarily denote - say - 
political extremism, nor we speak of "moderates" to identify the center of the opinion 
space. 

To keep things simple, we consider only the unidimensional case, i.e. the state of 
an agent is a scalar number in the interval [0, 1]. We denote with x (t) the state of a 
generic user at time t and with y (t) the state of a generic page. 

The interaction rule between a user and a page captures the dynamics of social 
influence. Let us imagine that at time t a user edits a page. Let fi € [0, l /i] be the speed 
(or uncertainty) parameter and e e [0, 1] the confidence IT8l . If \x (t) — y (t)\ < e 
then: 



x{t) «- x(t)+n(y{t)-x(t)) (1) 

y(t) «- y(t)+fi(x(t)-y(t)) (2) 

else, if \x (t) — y(t)\ > e, we allow only Eq. |2) to take place with probability r. 
This addition to the bounded confidence averaging rule reflects the fact that, in peer 
production systems, users often deal with content they do not agree with without being 
influence by it, as when a vandalized page is reverted to a previous, non-vandalized 
revision (also known as rollback). 

Different pages can reflect different topics and hence receive attention from users 
based on their popularity. We employ a simple reinforcement mechanism to model 
this. Let c p > be a constant. If m t is the number of edits a page has received up to 
time t, then the probability of it being selected at that time will proportional to m t + c p . 
When c p — > oo, pages will be chosen for editing with uniform distribution, regardless 



of the number of edits they have received. Hence, we can study the impact of of content 
popularity in user participation by setting c p to a small or large value. Of course users 
do not always choose to edit an existing page. Sometime, a user can decide to create a 
new page. We model this by considering a rate of new page creations p p . Whenever a 
new page is created, its state y is equal to the state x of creator. Creators are chosen at 
random among existing users. 

In order to model user participation, the population of users is dynamic. First, we 
consider an input rate of new users p u , whose state is chosen at random within the 
interval [0, 1]. Second, we consider a inhomogeneous departure rate that depends on 
the experience of users. Let us consider a generic user at time t and let us denote with 
n t the number of edits he (or she) did up to t, and with s t the number of these edits that 
resulted in the application of Eq. ([TJ. Let c s > be a constant and r(t) be the ratio 

r(t)= S ^ (3) 

n t + c s 

The rate of departure Xd (t) is then defined as: 

MO-^ + i^M (4, 

to n 

with To 3> T\ time scale parameters. Depending on the value of r (£), the expected 
lifetime (r) will interpolate between two values: (r) = tq (long lifetime) for r (t) — 
1, (r) = n if r (t) = (short lifetime). If c s —> oo, we recover a homogeneous 
process with rate Tq , so we can set c s to control how sensitive the departure rate is to 
unsuccessful interactions. 



3 Evaluation Methods 

3.1 Computer Code Emulation via Gaussian Processes 

Although we can perform the statistical evaluation of our peer production model us- 
ing directly the computer simulator, this approach is not desirable, as evaluation of 
the computer code can be quite time consuming. We rely instead on emulation of the 
computer code output. We use a Gaussian Process (GP) as a surrogate model of the 
average lifetime (r) of users in our peer production system. Gaussian processes (or 
Gaussian Random Functions, GRF) are a supervised learning technique used for func- 
tional approximation of smooth surfaces and for prediction purposes: see [25| for the 
application of GP to computer code evaluation. 

Given input sites o b s = (#i, 82, ■ ■ ■ , On) we can evaluate our model as specified 
above, and obtain observations of the average user lifetime T b s = (ti,T2, • • . ,tjv). 
Based on these observations, we wish to predict the value of r at an untested input site 
6, i.e. t (6). With a GP, this value is t (6) — E [r (9) \ 9 obs ]; the uncertainty in the 
prediction, that is, Var [f {$)], is equal to Var [r (6) | o bs]- With it we can compute a 
confidence interval that characterizes the uncertainty of the prediction of r based on 
training data (9 ob s, T bs)- 



There are several strategies for selecting the input sites 9 b s at which we will run 
our computer simulator. Here we choose to employ a uniform, space-filling design 
generated via Latin Hypercube Sampling (LHS) because it yields better error bounds 
than those produced with uniform random sampling |fT9l . The space-filling require- 
ment is attained using a maximin design. A maximin design is any collection of points 
6 that maximizes the minimum distance between points: max© min^/ 0j — L i || 

3.2 Global Sensitivity Analysis 

A computational or mathematical model is comprised usually of a number of parame- 
ters, or factors, which are meant to affect in some way its output, or response. Hence, 
in general, a model can be thought as a mapping between factors (input) and responses 
(output). One might be interested in the problem of quantifying how much output "vari- 
ability" in this mapping can be apportioned to each of the inputs. Global Sensitivity 
Analysis (GS A) is a set of statistical techniques used to get an answer to this problem. 
See |24l for a primer on GSA. 

One application of GSA is factor screening. The ranking of parameters is usually 
done by computing the sensitivity indices of each input parameter (factor). There are 
various techniques for computing the sensitivity indices, each with its own properties 
and assumptions. In this study we computed sensitivity indices by decomposing the 
output variance of our surrogate model. We used other techniques as well, namely 
partial correlation coefficients and standardized regression coefficients, and they gave 
concordant results. We choose to report here only the results of the decomposition of 
variance because it applies more naturally to non-linear models like ours. 

The method we use was proposed by Sobol' and is based on the analysis of vari- 
ance (ANOVA) (28). The idea is to decompose the variance of the output in several 
components that are attributable to independent factors, in our case the parameters of 
the model. 

Let us assume that the space of parameters is [0, l] , where d is the number of 
parameters. SoboL proposes to write the output Y as: 



Y(e 1 ,...,e d )=Y + J2M e i)+ E y iA e iA)+ 

i=l l<i<j'<d 

+ '--+Yi,2,... ld (9i,0 2 ,..-,O d ) (5) 

and shows that this decomposition is unique under the assumption that components are 
orthogonal and have zero mean. In Eq. |5]), Yq = E [Y], Yi(9i) is the main effect 
of parameter 0$, Yi^{d%,6j) is the 2-way interaction effect between the i-th and j'-th 
parameters (i ^ j), and so on. Each summand is computable from suitable integrals. 
For example the main effect of Yi is: 

Yiifii) =[■■•[ Y ^- ■ ■ . e d)dO^ - Y (6) 

Jo Jo 

where with O^i when mean the reduced parameter vector obtained by considering all 

parameters except 6*j. Similar formulas can be obtained for higher order effects. Let 



Table 1: Parameters settings for global sensitivity analysis. 



Parameter 


Variable name 


Symbol 


Value(s) 


Unit 


Distribution 


Const, popularity 


const_pop 


p 


(0,100) 




uniform 


Const, successes 


const_succ 


C s 


(0,100) 




uniform 


Confidence 


confidence 


e 


(0,'/2) 




uniform 


Daily edit rate 


daily _users 


Ae 


(1,20) 


day 


uniform 


Daily rate of pages 


daily _pages 


Pp 


(1,20) 


'/day 


uniform 


Daily rate of users 


daily _edits 


Pu 


(1,20) 


1/day 


uniform 


Initial no. of pages 




Np 






see Subsec. 


4.2 


Initial no. of users 




N v 






see Subsec. 


4.2 


Long lifetime 


longJife 


TO 


(10,100) 


day 


uniform 


Rollback probability 


rollback_prob 


r 


(0,1) 




uniform 


Short lifetime 


shortJife 


n 


(1/24,1) 


day 


uniform 


Simulation time 




T 


1 


year 




Speed 


speed 


/'■ 


(o, 1 /*) 




uniform 


Transient time 




To 


2 


year 





us now consider the variances of the summands of Eq. |5]). We can decompose a 2 , the 
total variance of Y, as: 



£*? + £ - 



'1.2,. ..,<* 



(7) 



l<i<j<d 



The sensitivity indices proposed by Sobol' are obtained by standardizing all sum- 
mands of Eq. |7]), obtaining: 



l<i<j<d 



(8) 



8=1 



Mi is the main sensitivity index of parameter 9u Cij is the two-way interaction index 
between 6i and 0j, etc. Two quantities are of interest for assessing the importance of a 
parameter: the already cited main sensitivity index Mf, and the total interaction index 
Ti, which is defined as the sum of all terms that involve parameter Be 



Tj = / y Cj t j + 2 , Cj,j,fe + •• • + Ci ) 2,...d 



j¥' 



l<j<k<d 



(9) 



4 Results 



4.1 Simulation scenario 

TablefTllists all parameters of the model, together with simulation settings. Two quan- 
tities have been held fixed: simulation time, and transient time. Two other parameters, 
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Figure 1 : (a) transient time determination, (b) main effects plot 



the initial number of users N u and the initial number of pages N p , are determined after 
a transient, see Subsec. 4.2 below. All remaining parameters, instead, were assigned a 
range of values. To sum up, we had an input space of 10 independent dimensions. 

We chose the long (to) and short (n) user lifetimes to range in non-overlapping 
intervals corresponding to different time scales, consistently with empirical observa- 
tions of user participation from Wikipedia [7|. The value of the simulation time T was 
chosen so that a simulation would comprise more than one generation of long-term 
users. 

Intervals for event rates such as the daily rate of edits (A e ), of new user arrivals 
(p u ), and of new page creations (p p ), were chosen looking at plausible values from 
the public statistics on the Wikipedia project^ These parameters have a strong influ- 
ence on simulation time, therefore ranges for them were set trying to strike a balance 
between exhaustiveness of the sensitivity analysis and simulation wall clock time. 

The choice of ranges for the constant popularity term (c p ) and for the constant 
successes term (c s ) was a bit more problematic. To our knowledge, none of them has 
ever been studied before in the context of peer production communities. We settled for 
ranges we deemed would be large enough for our purposes. 

Finally, the opinion dynamics parameters. It is clear that \i < l /i. Regarding the 
confidence e, the literature on bounded confidence models in one dimension suggests 
that for e > [ /2 the dynamics of consensus does not change noticeably. This should 
apply also to the dynamics of user participation in our model. We ran some simulations 
of the average lifetime, and found confirmation to this intuition. We thus restricted e to 
the interval (0, V 2 )- 



4.2 Transient 

Transient duration To was determined empirically: we plotted the daily number of 
users N u (d; 9), d = 1, 2, . . . , for various values of the parameters 9 and chose Tq 



These statistics are freely available on http://stats.wikimedia.org 



as the time after which all curves look stationary. Figure la reports the results of this 
exercise. In the figure, the shaded region corresponds to the transient interval (0, To). 
The value of To is 730 days. The values of 9 were taken from a maximin LHD with 
50 points. Each curve is scaled by its average value N u (9) computed over the interval 
d £ [731, 1095]. The yellow solid line is a B-spline fit of 50 evenly spaced observations 
of the expected scaled number of users n ^[n u , and serves as a guide for the eye. 

During the transient phase we did not record any data, so that the estimation of r, 
on which the sensitivity our analysis is based, did not reflect the dynamics of opinion 
formation during the transient. 

4.3 Factor screening via global sensitivity analysis 

We sampled a maximin Latin Hypercube Design (LHD) with 50 points using the inter- 
vals listed in Tab. u\ To sample a decent maximin design, we generated 10 4 hypercubes 
at random and selected the one that maximized Eq. ( |3.1[ ). We computed the average 
user lifetime (r (9)) by running 10 replications for any 9 and averaging the values 
obtained. 

We first plotted the values of the response variable r versus each input parameter 
to check visually for any linear trend. Scatter plots are shown in Fig. [2] A multiple 
linear regression gave a coefficient of determination R 2 = 0.83. However, no clear 
trend emerges from the plots for all parameters except for the confidence e and the 
long lifetime To. For the latter, something similar to a linear trend can be seen, whereas 
for the other the relationship looks more of sigmoidal type. We tried fitting a sigmoid 
function to r as a function of e. The result of a K-S test (p-value < 3.5 x 10~ 4 ) 
rejected the normality of the residuals, and therefore led us to exclude a sigmoid model 
as a possible functional form of r (e). 

Next, we fitted a GP emulator to the average user lifetime data, using the open 
source machine learning toolkit from the SciKits collectiorr] We then discarded the 
simulator and used f (9) in lieu of it. To compute the sensitivity indices we used the 
Winding Stairs (WS) method, a resampling technique proposed in fl4l . We computed 
main (Mi) and total interaction (T) effect indices for each parameter (i = 1 . . . 10) 
using a WS matrix with 10 4 rows. The results are shown in Tab. [2] 

The total variance a 2 was also computed from W (each column of a WS matrix is 
an independent sample). The WS method yields better estimates of the total interaction 
effects than other methods [5|, so we impute the presence of some slightly negative 
values of AT to the uncertainty in the estimation of the total output variance a 2 and to 
the presence of factors with almost null total effect. 

Only two factors have a T > 3%. These are the confidence e, and the long term 
lifetime r . We explored further the individual contribution of each parameter in the 
output variance by looking at the main effect plots. These are plots of Y (0$) as a 
function of 8i, and can be obtained evaluating Eq. |6]l using Monte Carlo averaging 
and the GP emulator. To facilitate comparison of the different parameter ranges, in 
Fig. fTblwe plotted the main effect as a function of the scaled parameter value. Figure 



Home page: http://scikit-learn.sourceforge.net/ 



daily _edits daily .users daily _pages 



confidence 




0.00 , 0.25, . 0.50 
snort _life 



-o 
o 



m 



1 o 



0.0 



0.5 



O 



■° o ° 

cPo Q(9 £o 
- 0,00 &> q 




10 20 

const _succ 



10 20 0.00 0.25 0.50 

const_pop rollback_prob 




, 50,. „ 100 
long-life 



& 






1.0 10 



55 



100 



Figure 2: Scatter plots of (t) versus 9 = (A e , p ui p p , e, fi, c s ,c p , r, t , n). Error bars 
(standard error of the mean lifetime computed over 10 realization) are all smaller than 
the data points. 



lb shows that p p and t\ have a slight effect on user lifetime too, the first negative and 
the second positive. 

The difference between Ti and Mj is the fraction of variance that is only due to 
interactions between 6i and any other parameter or groups of parameters. For e this 
difference is 0.08 and for tq it is 0.05. Summed up together, this residual interaction 
effect amounts to almost three quarters (77%) of the total interaction effects from all 
remaining parameters. Thus we expect e and r to have some interesting interactions 
with other parameters. We explored two-way interactions systematically using two- 



way interaction plots, which are the 3D counterparts of the curves of Fig. lb 

Given two parameters 6$ and 9j, with i ^ j, we computed YijfOi, 9j): we eval- 
uated Eq. ^ in a similar way, this time holding fixed the values of two parameters 
instead of one. Here we report the results on the interaction between e and other pa- 
rameters, included to. The plots are shown in Fig. [3]and|4] 

Almost all parameters show just a weak interaction with e, which occurs at low 
(e < 0.1) and high (e > 0.4) values of it. Only the pair {e, tq} shows a significant 
degree of interaction. 



4.4 User lifetime distribution 

Previous studies on continuous opinion dynamics under bounded confidence show that, 
as e grows, the population of agents undergoes a gradual change from a regime with no 
consensus, to a regime of total consensus with a single cluster (9JH21 • In our model this 



Table 2: Variance decomposition. Winding Stairs sample size 10 rows, total variance 
635.365 days 2 . 

Parameter M { T, 

A e -0.002 0.014 

p u -0.003 0.02 

Pp 0.003 0.027 

e 0.65 0.73 

H -0.004 0.03 

c s 0.004 0.03 

c p -0.005 0.016 

r -0.005 0.026 

n 0.002 0.03 

r 0.18 0.23 



shift must reflect somehow in the average user lifetime, but what shape the user lifetime 
distribution takes during it? The findings from the previous section let us restrict the 
field of study to just two parameters of the original ten, namely e and tq. In this section 
we focus only on them, and try to understand what is the actual distribution of user 
lifetimes, by simulating from our model. 

We performed simulations holding fixed the user lifetime parameters (to = 100 
days and t\ = 1 hour), while changing the value of e. The values of all other pa- 
rameters were fixed to the midpoints of the respective ranges listed in Tab. [T] We 
computed the log-lifetime u = log (r) and fitted a 2-components Gaussian Mixture 
Model (GMM) to u. Figure Blreports the result of the fitting, showing the densities of 
the individual components using stacked area plots. We report here only two values 
of e, e = and e = 0.3, which is a value greater than the threshold for consensus in 
Deffuant's model, to show the difference between the two regimes. 

5 Discussion 

In this section we discuss the main findings of the present study. We presented an 
agent-based model of user participation in a peer-production community. We model 
participation as a bounded confidence consensus process, where users modify content 
according to their objectives and skills (represented by a continuous state x), and are 
in turn indirectly influenced by the rest of the community. We use global sensitivity 
analysis to study the importance of the model's parameters in explaining the average 
user lifetime. The first interesting - and rather surprising - finding is that, as shown 
in Tab. [2] of the overall ten parameters of the model, only two affect the average user 
lifetime in a considerable way. This is interesting because it suggests that several other 
factors like content popularity, user community growth, and user activity rate, are not as 
important as the general level of "tolerance" of the community (given by the confidence 
e) in affecting the process of group consensus. Moreover, interaction plots show that 



10 



relevant interactions occur between e and tq: this confirms the intuition that the role of 
To is to set the support of the distribution of r, and that e acts as a switch, controlling 
the transition from a regime where only short-term forms of participation are possible, 
due to the low rate of successful user-page interactions, to a consensus regime where a 
cluster of long-term users is able to emerge. 

Of course, the results from the factor screening should be also viewed in light of 
our simulation setup. We decided to focus on a stable community, where the num- 
ber of users N u is stationary, and not on the initial phase of community formation. 
Plausibly, during this transient phase other parameters, such as the speed /i, and the 
rollback probability r, might have more importance in determining the span of user 
participation. 

The second interesting finding is about the actual distribution of user participation, 
which is markedly bimodal. From Fig. Blit is possible to appreciate, for e = 0.3, a clear 
subdivision in two groups of users based on their participation span. We can see also a 
subdivision for e = 0, which is probably related to the fact that c s = 50 in that setup. 
Although we did not perform a proper model calibration, this finding is encouraging, as 
previous studies on the distribution of user accounts lifetime in Wikipedia have shown 
a similar bimodal pattern [7 1. 

In general, both findings show that agent-based model can be studied through the 
systematic use of simulations and computer code emulation, and provide a novel con- 
nection between model of opinion dynamics, whose study has been so far notoriously 
lacking on the empirical side l4l l27l . and peer production. 
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Figure 4: Two-way interaction plots (cont'd) 
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Figure 5: GMM fit of log-lifetime of user accounts in two different runs of the model. 
For e > e c a bi-modal pattern is a clear feature of user participation. 
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